Skip to content

Zeppelin-275: Add SpringXD Stream and Job Interpreters #291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
tzolov wants to merge 11 commits into from
Closed

Zeppelin-275: Add SpringXD Stream and Job Interpreters #291

tzolov wants to merge 11 commits into from

Conversation

@tzolov
Copy link
Contributor

@tzolov tzolov commented Sep 8, 2015

1. SpringXD Stream Interpreter:

  • %xd.stream - identifies the SpringXD Streams interpreter.
  • Use (Ctrl+.) for auto-completion.
  • Streams must have names. Follow the convention: (stream name = stream definition). Streams without names are ignored.
  • New Run destroys any previous streams created in the same paragraph. The previous streams are destroyed even if their names have been changed or removed.
  • New Run will Create and automatically Deploy all streams defined in the Paragraph.
  • If one stream fails to create or deploy all streams in the paragraph are destroyed
  • Zeppelin returns after the streams deployment (e.g. Zeppelin goes into Finished state).
  • When streams have successfully been deployed the result contains a button that lists the just deployed streams and status 'DEPLOYED'
  • To destroy streams in the paragraph press the Annular button in the result section. Button state should switch from 'DEPLOYED' to 'DESTROYED'.
  • Streams can refer other streams or jobs in any paragraph and even different notebooks

2. SpringXD Job Interpreter:

  • %xd.job - identifies the SpringXD Job interpreter.
  • Use (Ctrl+.) for auto-completion.
  • Jobs must have names. Follow the convention: (job name = job definition). Streams without names are ignored.
  • New Run destroys any previous jobs created in the same paragraph. The previous jobs are destroyed even if their names have been changed or removed.
  • New Run will Create and automatically Deploy and Start all jobs defined in the Paragraph.
  • If one job deployment fail all jobs in the paragraph are destroyed
  • Zeppelin returns after the jobs have started (e.g. Zeppelin goes into Finished state).
  • When job have successfully been deployed the result contains a button that lists the just deployed jobs and status 'DEPLOYED'
  • To destroy all jobs in the paragraph, press the Annular button in the result section. Button state should switch from 'DEPLOYED' to 'DESTROYED'.
  • Jobs can refer other streams or jobs in any paragraph and even different notebooks

@tzolov tzolov force-pushed the ZEPPELIN-275 branch 3 times, most recently from c7d38ba to 6774d15 Compare September 13, 2015 12:04
@bzz
Copy link
Member

bzz commented Jan 5, 2016

Looks like this guy diverged from master as well as having some commits that are not part of ZEPPELIN-275

@tzolov
Copy link
Contributor Author

tzolov commented Jan 5, 2016

Good spot @bzz, I will try to clear the ghost commits.

@tzolov
Copy link
Contributor Author

tzolov commented Jan 15, 2016

@bzz, i've resolved the commits mix-up, please review.
Also i've rebased the ZEPPELIN-275 branch to the master.

@bzz
Copy link
Member

bzz commented Jan 15, 2016

Great!
Quick question - do we need to add this interpreters to the list of interpreters in docs? (They now live in master, under ./docs)

Also, as they become part of releases - are there any new dependency that it brings, that need to be added to zeppelin-distribution/src/bin_license/LICENSE ?

@tzolov
Copy link
Contributor Author

tzolov commented Jan 15, 2016

  • Sure the interpreter should have documentation - I will add such to the ./docs
  • The interpreter brings 2 deps: spring-xd-rest-client (under Apache2 License) which i will add to the LICENSE file and Guava - already there

@bzz
Copy link
Member

bzz commented Jan 16, 2016

Sounds great, thank you very much!

On Sat, Jan 16, 2016 at 3:14 AM, Christian Tzolov [email protected]
wrote:

  • Sure the interpreter should have documentation - I will add such to
    the ./docs
  • The interpreter brings 2 deps: spring-xd-rest-client (under Apache2
    License) which i will add to the LICENSE file and Guava - already there

—
Reply to this email directly or view it on GitHub
#291 (comment)
.

Kind regards,
Alexander.

@tzolov tzolov force-pushed the ZEPPELIN-275 branch 2 times, most recently from fde1cba to 6b746ea Compare January 17, 2016 16:39
@tzolov
Copy link
Contributor Author

tzolov commented Jan 17, 2016

Cheers @bzz,
I've updated the LICENSE file with the springxd dependecy. Also i've put together a basic springxd documentation page and listed it in the interpreter's menu. Let me know if we are still missing tasks.

AhyoungRyu
AhyoungRyu reviewed Jan 18, 2016
View reviewed changes
docs/interpreter/springxd.md Outdated
Copy link
Contributor

@AhyoungRyu AhyoungRyu Jan 18, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that back quotation marks are not working in the table tag. Like this below image.
screen shot 2016-01-18 at 5 53 55 pm

So, why don't you use <code> tag instead of the back quotations? : )

@tzolov
Copy link
Contributor Author

tzolov commented Jan 18, 2016

Thanks @AhyoungRyu! I've just commited a fix for the abovementioned misspellings.

bzz
bzz reviewed Jan 19, 2016
View reviewed changes
conf/zeppelin-site.xml.template Outdated
Copy link
Member

@bzz bzz Jan 19, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were == and < intentionally added here?

@bzz
Copy link
Member

bzz commented Jan 19, 2016

Looks great to me, thank you! Just needed to be updated on top of the latest master.

@tzolov tzolov force-pushed the ZEPPELIN-275 branch 3 times, most recently from 600707c to 9102439 Compare January 27, 2016 16:12
@corneadoug
Copy link
Contributor

corneadoug commented Sep 27, 2016

@tzolov @bzz
Do we want to rebase and try to merge this PR?

@asfgit asfgit closed this in c38a0a0 May 9, 2018
asfgit pushed a commit that referenced this pull request May 9, 2018
@jongyoul
Close outdated PRs
b11d355 
close #83
close #86
close #125
close #133
close #139
close #146
close #193
close #203
close #246
close #262
close #264
close #273
close #291
close #299
close #320
close #347
close #389
close #413
close #423
close #543
close #560
close #658
close #670
close #728
close #765
close #777
close #782
close #783
close #812
close #822
close #841
close #843
close #878
close #884
close #918
close #989
close #1076
close #1135
close #1187
close #1231
close #1304
close #1316
close #1361
close #1385
close #1390
close #1414
close #1422
close #1425
close #1447
close #1458
close #1466
close #1485
close #1492
close #1495
close #1497
close #1536
close #1545
close #1561
close #1577
close #1600
close #1603
close #1678
close #1695
close #1739
close #1748
close #1765
close #1767
close #1776
close #1783
close #1799
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@tzolov @bzz @corneadoug @AhyoungRyu